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ABSTRACT 

The paper considers nonlinear time series whose second order 
autocorrelations satisfy autoregressive Yule-Walker equations. The usual 
linear residuals are then uncorrelated, but not independent, as would be 
the case for linear autoregressive processes. Two such types of nonlinear 
model are treated in some detail: random coefficient autoregression and 
multiplicative autoregression. The proposed analysis involves 
crosscorrelation of the usual linear residuals and their squares. This 
function is obtained for the two types of model considered, and allows 
differentiation between models with the same autocorrelation structure in 
the same class. For the random coefficient models it is shown that one 
side of the crosscorrelation function is zero, giving a useful signature of 
these processes. The non-zero features of the crosscorrelations are 
informative of the higher order dependency structure. In applications this 
residual analysis requires only standard statistical calculations, and 
extends rather than replaces the usual second order analysis. 
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1. INTRODOCTIOII 

This paper is concerned with the statistical assessment of dependency 
beyond autocorrelation in the context of nonlinear time series models. The 
central theme is that residuals from models fitted according to 
inapplicable linearity assumptions can also profitably be used for further 
analysis beyond linearity. A well known property of linear time series 
residuals is that they are uncorrelated; in the context of linear models 
they should also be independent, apart from the effects of parameter 
estimation with short series. However, in the context of nonlinear models 
it is not often recognised that uncorrelated residuals also hold 
information concerning higher order dependence in the data. [developments 
of such a higher order residual analysis will be explored here, in 
particular for two types of nonlinear autoregressive model which have the 
usual linear Yule-Walker autoregressive correlation structure. Some higher 
order dependency correlations will be obtained. In a companion paper, the 
suggestion of reversed residuals will be made, and the analysis given in 
this paper will be extended to encompass these reversed residuals. 

2. AUTOREGRESSION AND LINEAR AlfPOREGRESSIVE RESIDUALS 
2.1 Autoregression for Nonlinear Hodels 

The standard form of autoregression needs widening for use with 
nonlinear models; we consider first the standard form which is explicitly 
autoregressive in a linear additive way and then several weaker variants. 
A stationary time series {X^} of mean fi is assumed. Under the Itnear 
autoregressive model, of order p, the {Xt) satisfy the equation 

Xt-^=ai(Xt_i-^i)+a2(Xt:_2-'M)+. - .+ap( Xt-p-M-)+<st ^ t-0, ±i, ±2, . . . (2.1) 

where the are independent and identically distributed and ^, 0 ^, 02 ^ ... /Oip 
are fixed parameters. A more general definition of autoregression of order 
p, could be the linear conditional expectation requirement that 
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E( pt I X-^-.j_ / X-^_2 0 * i X-^— p ) 

= ai(Xt-l“^H^2(^-2“M)+- • •+0Cp(Xt-p~4)/ t*0,±l,±2, . . . . (2*2) 

The definition (2.1) ijnplies (2.2) but not vice-versa. Thus this 
definition could apply to models which are not of the linear form (2.1)^ 
either because the are dependent^ but still with 

E( 6^ |X^_l,Xt-2' • • • because the model hats some other structure 

altogether. For instance/ there are the random coefficient models of 
Nicholls and Quinn (1982)/ the exponential distribution random coefficient 
models of Lawrance and Lewis (1981/1984)/ the discrete distribution random 
coefficient models of Jacobs and Lewis( 1983 )/ and the gamma-beta random 
coefficient models of Lewis (1981). 

Random coefficient autoregressive models of order p take the general 

form 

Xfc = At< Xt-1+At<2)xt_2+. • .+At^P^Xt-p+Bt, t=0, ±1, ±2, . . . (2.3) 

where the vector of coefficients {A-^^ ^ A^^ ^ / A^^ P), } is a stationary 

vector sequence of independent random variables, and sometimes in addition, 
Bt is independent of the random coefficients at time t. It is eeisy to see 
that such models satisfy the linear conditional expectation definition of 
pth order autoregression/ but are nonlinear. A first order case of the 
type (2.3) will be used to illustrate the proposed method of residual 
analysis of autoregressive nonlinearity. 

A further and weaker definition of autoregression is the requirement 
that the autocovariances of the (Xt)# denoted by (yk)/ just satisfy 
Yule-Walker linear difference equations of the form/ 

Vr=<*iyr-l’*‘«2yr-2'*‘ • • • +<^pyr-p^ yr=y-r ^ . 2 / . . . (2.4) 

for suitable constants ai/a 2 / . . . /Otp; this will be referred to as 
Yule'Ua'Lher autoregression. It is true for processes which satisfy (2.1) 
and (2.2)/ as may be verified in the usual manner by multiplying X^ by X^_^ 
and taking expectations. The reverse is not true, (2.4) does not imply 



(2,1) or (2.2). A case in %4ilch (2.4) holds but in <4iich (2#1) and (2.2) 
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do not, is the product autoregression model of McKenzie (1982), where, with 
p=l, E(XtlXt~i) is a fractional power of Xt-i; this model will also be used 
in Section 4,3 as an ilustration of the proposed residual methods. 

A variety of other models can satisfy the Yule-Walker definition of 
autoregression, and yet not satisfy the linear conditional expectation 
definition. Amongst these are first order Markov chains under fairly 
general conditons, and others such as the semi-Markov generated processes 
discussed in Cox and Lewis (Chapter 7, 1966) and Lewis (1980). Notice that 
we do not define nonlinear autoregressive models in a constrxictive way. 
The class is so wide as to make this impossible; one such class has been 
studied by Jones (1978). Rather, we require that the autocorrelations 

should satisfy linear ecpjations, similar in structure to those satisfied by 
the autocorrelation of linear autoregressive models. In view of this, our 
suggested analysis extends, rather than superceeds, conventional methods. 

2.2 Definition and Discussion of Linear Autoregressive Residuals 

For the aina lysis of time series data involving models satisfying (2.1), 
or (2.2) or minimally (2.3), the use of Itnear autoregressive residuals of 
order p, defined as 

Rt< P ) * ( XfM )-ai( )-a2< Xt-2~M )~ • • • -otp( ) (2.5) 

is suggested. This suggestion is baised on the following theorem, which is 
a generalization of a result given in Lawrance and Lewis ( 1984, Section 
7.2) for p=2 . 

Theorem. Let the stationary process {X^} satisfy the Yule-Walker type 
equations (2.4). Then the linear autoregressive residuals {Rt^^^} defined 
at (2.5) are uncorrelated (although not necessarily independent). 

Proof. The autocovariances of the residuals (2.5) are 
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Cov[Rt( P > , Rt+r^ P ^ ] = Cov[ ( Xt-ti ) . Rt+r^ ^ ^ ]-aiCov[ ( , Rt+r^ P > ]- . . . 

-apCov[(Xt_p-M),Rt+r^Ph (2.6) 

= Covt(5Ct-/x),Rt+r^PM-aiCov[(Xt-M),Rt+r+l^P^]-- • • 

-apCov[(Xt~M)/Rt+r+p^^^]- (2.7) 

Ec[uation (2.7) follows because the {X^) process is stationary and 
consequently the process is stationary. The covariances in (2.7) 

need only be considered for positive lag since the autocovariance is an 
even function of r. Then the crosscovariances on the right-hand side of 
(2.7) are all of the same type and given by 
Cov[ ( Xf^ ) , Rt+r^ P h = Cov( ( Xfji ) , { ( xt+r~M >-ai( xt+r-i~M ) • • • ~ap( Xt+r-p'M ) ) ] 

= yr-«iyr-l- • • --Opyr-p r=l, 2, .... (2.8) 

Now by the Yule-WaUcer equations (2.4), the expression (2.8) is zero. Thus 
using (2.8) in (2.7) 

Cov[Rt<P^,Rt+r^P^ = 0 r=±l,±2,..., (2.9) 

as was to be proven. The proof is immediate for the linear autoregressive 

model (2.1), since and 6^ and R^+r^^^ definition 

independent. 

Note that the linear autoregressive residuals will generally 

still be dependent though uncorrelated in nonlinear modelling of the (X-t); 
with the linear autoregressive model (2.1) the will not only be 

uncorrelaled but also independent. It ts thts difference which will he 
exploited to explore nonlinearity In pth order nonlinear autoregressive 
processes . The dependency attributable to parctmeter estimation is taken to 
be small in the large scale applications we have in mind; indeed, nonlinear 
modelling of short series of data may well be hard to justify. 

The (Quantities are autoregressive residuals in the sense that 

is the residual of X^ after subtracting off its best linear 

least s<Quares predictor in terms of X^-i, X^-2^ • • * ^^-p^ given by 

^(P) = ^+ai(Xt_i-M)+a2(Xt-2-/i)+*--+«p(Xt-p-M)- (2.10) 

*nwi8 the tealdualB qlve the basic way of taxing out the linear 
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correlatjon component in models with pth order autoregression minimally of 
the *Yule-WalJcer* form (2.4). Note too that if the process is 
autoregressive in this sense, linear autoregression on terms beyond 
for example ( yield <Xp^y^^O, k=l,2,...; this is a property of 
the linear component being taken out so as to minimize the expected meain 
square of the residual 

A further point worth noting about the residuals concerns their 

crosscovariances with the X^^’s. In the proof of the theorem it is seen 
that Cov[Xt“iLi,Rt+r^^^l “ 0 r = 1,2,... . However, the other half of 

these crosscovariances is non-zero. 

Exaa^le: The nonGaussian linear AR(1) model 

Taking (2.1) with p=l,and p instead of a^, the crosscorrelation of X^-M aind 
in the AR(1) model is given by 

c:orr( X-t-;A/Rt-r ) “ ( 1-p^ )^/^ p^ for r=l, 2 , 3, . . . ; ( 2 . 11 ) 

the superscript hais been dropped from Rt-r^ i*^ will be in similar future 

use. Note further, that in this case Rt+r i® independent of X^ for 

r=l,2,... . Further use of (2.11) will be made in Sections 3 and 5. 

For the use of in data analysis, the order p of the linear 

cLspect of the autoregression needs to have been chosen; any of the 
available standard methods may still be used. In addition, of course, the 
)j.,ai,a 2 , . . . ,ap need to be estimated; there are at least two convenient 
possibilities: (1) the assumption of a linear autoregressive model like 

(2.1) and the use of least squares estimation, and (2) a non-model bcused 
approach to estimation employing the first p Yule-Walker type equations 
(2.4). The latter is suggested here; however, Tjostheim aind Paulsen (1983) 
recommend (1) when dealing with modest sized samples from linear models, on 
account of serious estimation bias with the Yule-Walker estimates. Since 
the use envisaged here is primarily with nonlinear models and substantial 
sets of data, the superiority of ( 1 ) over ( 2 ) is not established or 
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crucial. 

3. ASSESSMENT OF HIGHER ORDER DEPENDENCE 

Since the linear autoregressive residuals {Rt^^^} of (2.5) have zero 
covariances when (2.4) holds, but for nonlinear processes need not be 
independent, a residual analysis of this nonlinearity can be based on an 
assessment of their higher order dependence. To consider what form this 
might take, we note that the use of [X^^] has been suggested by Granger and 
Andersen ( 1978, p. 63) for bilinear models, for which many of the simple 
cases have zero autocorrelations themselves. The corresponding initial 
suggestion here is, as was briefly illustrated in Lawrance and Lewis 
(1984), to use the residuals {(Rt^^^^)* Displays can then easily be made 
of the associated autocorrelation functions, scatter plots, periodograms, 
cumulativo periodograms, etc, using standard (second order) time series 
software . 

However, the autocorrelations of {(Rt^^^)^} fourth order quantities 
in the original series (X^), which is a double jump from the second order 
autocorrelations of the series (X^). Such quantities will be very 
difficult to handle theoretically with roost types of nonlinear model. 
The crosacorrelatton function of {Rt^^^} and is essentially 3rd 
order, but by involving the variance of {(Rt^^^)^}/ also needs some fourth 
order joiiiL moments of {X^} up to lag p. The necessary calculations of all 
these quantities for two nonlinear models will be given in Section 4. 

The behaviour of the crosscorrelation function of Rt^^^ and {(Rt^^M^) 
may be judged against the fact that the {Rt^^^} are tndependent for the 
linear autoregressive model of order p, and hence it will be zero except at 
lag zero. For the random coefficient autoregressive processes considered 
in Section 4, the suggested crosscorrelation function will be shown to 
posess a useful cut-off property; this generalizes the N£AR( 2 ) result given 
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in Lawrcuice and Lewis (1984). 

Two rather simpler crosscorrelation quantities cam be proposed from 
consideration of the zero covariance result of (2.8 ), and the additional 
fact that when the pth order linear autoregressive model holds^ Xt and 
Rt+r(^) (r-1^2,...) will be independent, and not just uncorrelated. 

Working in terms of {Xj.} adjusted for its mean /x, which is better 
computationally and often nicer theoretically, the following may be 
considered 

Corr[(Xt“M)^#Rt+r^^^]^ r=0, ±1, ±2, . . . (3.1) 

Corr[Xt-/x# ( Rt+r^^^ r=0±l,t2,... . (3.2) 

These autocorrelations are not ecjuivalent in the aspects of higher order 
dependency of th<i (X^) process which they assess. To see this, note that 
the cova,riajice corresponding to (3.1) involves only third order joint 
moments of the form E[(X|--/x)^(Xj;.+r-i-/x)], i=0, +1, . . . , tp while the 

covaricmce corresponding to (3.2) involves additional joint moments of the 
form E[Xt“/x)(Xt-|.r~/i)(Xt+r-i)]^ i=0, ±1, . . . , ±p. Further, the denominator of 
(3.2), by needing var{( more complicated in the higher order 

moments it involves relative to the denominator of (3.1) which needs 
var{Rt^P^}. However, both correlations, and particularly (3.1), are more 
tractable than the autocorrelations of {(Rt^^^)^} or the crosscorrelations 
of and {(Rt^^h^). 



Example (Continued from (2.11)): The nonCaunnian linear AR(1) model. 

As an Illustration of the use of (3.1) and (3.2), for the linear AR(1) 
model, ((2.1) with p=l and p Instead of ) there are the results 



Corrt(Xt-ti),(Rt-r)2] = 

fo 



.skew(X) 



Corr[(Xt-/i)2,Rt_j.] = 



Note 



3WX> ^ 

‘ ^[Var{(X-M)^)]^72 



1-/2 



r=0,l, . . . 
r=-l,-2, 

p^^/ X 0 / X / • , 



the fcLSter yeoiuetx ic decrease in (3.4) relative to (3.3 



(3.3) 



(3.4) 

(3.4) 



contcLins more higher moment infomtation. Olie other halves of both these 
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crosscorrelation functions are of course zero. Behaviour of this type for 
these functions - zero for r<0 and geometric decay for r>0 - would suggest 
linear modo.ln with nkowed marginal distributions or nonlinear models. 
Discussion of parallel results for two types of nonlinear model are given 
in section 4. 

Th(^ u«e of squaring in the construction of these higher order 
d('ipondency measures is recognized as being pragmatic and somewhat arbitary; 
it does however lead to expressions involving selected types of simpler 
higher order cross moments. The end use of the higher order dependency 
moaiuircM; can either bo exploratory for a given data set, to ascertain 
whethc'.r there is aippreciable nonlinearity present, or constructively, to 
provide evidence for fitting suitable typ<Mi of nonlinear model which cam 
match the observed highc^r order dependency. 

I 

Earlier discussion of higher order dependence in nonCaussian Linear 
models is givcm by Rosenblatt ( 1980) in terms of the bispectrum; aspects of 
nonlinearity and higher ordc^r sp<u:tra are briefly considered in Rosenblatt 
( 1979). 
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4. HIGHER ORDER DEPENDENCY FOR THO TYPES OF NONLINEAR MODEL 

In this section we obtain properties of the measures of higher order 
dependency discussed in the previous section^ for two specific types of 
nonlinear models. The models considered are autoregressive in the 
Yule-Walker sense of (2.4)/ but not in the linear sense of (2.1). 

4.1 Random Coefficient Autoregressive Models 

As already remarked in Section 1/ a generalization of the linear 
autoregressive model (2.1) is to let the coefficients ai/a 2 /...otp be random 
varicibles. One general class of such models has been discussed by Nicholls 
and Quinn (1982) who cite Andell (1976) and articles in the economic 
literature. Other classes of models with random coefficients include the 
discrete distribution models of Jacobs and Lewis (1983) amd the exponential 
models of Lawrance and Lewis (1981/1984). The class of random coefficient 
autoregressive processes to be of interest here is given by 

Xt ^ t=0/ ±1/ ±2 / . . . / (4.1) 

where is a stationary vector sequence of 

independent random variables with E(At^^ ^ )=<x j for j’^l^ . . . /P. The 

components of the vectors are not necessarily independent. For example the 
discrete distribution models of Jacobs and Lewis ( 1983 ) can be written in 
this form/ and have dependent coefficients/ as do the exponential models of 
Lawrance and Lewis (1981/ 1984). It is easily verified that the process 
(4.1) satisfies conditional expectation autoregression (2.2) and thus also 
the weaker Yule-Walker definition (2.4); it will also clearly be 
stationary. Note also that the standard linear autoregressive model (2.1) 
is a special case of (4.1) in which the random coefficients are actually 
constant. 

We now give a characteristic result for this type of process when 
higher order dependency is measured by Corr[Rf^(P) or 

)2]^ assuming that the R^-^P^*s are uncorrelated. 
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Otieorem . With the random coefficient model (4.1), 

Corr[Rt^P^,(Xt_r~M)^] and Corr[Rt<P>, (Rt-r^P^^] are equal to zero for 
r=l , 2 , ... . 

Proof : Using the definitions (2.5) and (4.1), 

R^( P ) = ( Xt-ix )-ai( Xt-i-M )- . . . -ap( Xt-p-M ) 

= ( At^ ^ ^-ai )Xt-i + ( At^ ^ ^-02 )Xt-2 + ••• + ( At^ P ^-Op )Xt-p 

+ Bt~( 1-ai. . .-ap)M . (4.2) 

On multiplying (4.2) by (Rt-r^P^)^ and taking expectations, thus obtaining 

Cov[Rt^P),(Rt-r^^^)^l' we have a sum of p terms given by 
P 

E E{(At<^)-aj)Xt-j(Rt-r^P>)2) (4.3) 

3=1 

euid a last term involving which is clearly zero. Now A^^ ^ ^-Cj is 
independent of both the X^-j and (Rt_r^P))2, for r=l,2,..., which may 

nevertheless be themselves dependent. Thus the jth term in (4.3) becomes 

E(A(3)-aj)E{Xfc_j(Rt-r^P>)2} = 0 for r=l,2 

since E(A(3))=«j. This completes the proof which clearly includes the 

first cross correlation mentioned in the theorem. The proof highlights the 
fact that tt ts the tndependence of the vector of coefftctents 

(At<l) ,At<P>,Bt} on prevtous X^'s uhtch creates the effect tn thts type 

of model. 

The results of the theorem can be used to help validate random coef- 
ficient autoregressive models; also useful in this respect are the non- 

zero higher order residual crosscorrelations, eg for positive r 
Corr[R^^P^,(Rt+j-^P))^]. These have been obtained for the second order 
autoregressive exponential process studied in Tjawrance and Lewis (1984); 
similar results for any first order random coefficient model of the type 
(4.1) are given in the next subsection. 

It is worth noting that with G^-r de^^ined as ajiy reasonable function of 
(Xt_^,Xt_r-i^ • • • )# 31 similar argument to that given in the proof shows that 



Corr[Rt^ 



r^I,2, . . . 



(4.4) 
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A result of this type is not, however, sufficient to establish, for 
instaince, that and are independent; they are dependent. 

The random coefficient autoregressive structure of (4.1) is not a 
necessary condition for the results of this section to hold; this may be 
illustrated by noting that Corr[Rt^P^,(Xt-r“M)^] is zero when 

Ci 2 (r )sE{(Xt-MkXt-r-M)^} satisfy the equations 

Ci2<r)=«iCi2(r-l)+a2Ci2(r-2)+. . .+apC] 2 <r--P), r=l,2,... . (4.5) 
The similarity of these equations to standard Yule-Walker equations (2.4) 
will be apparent. Thus any process with this property will have 
Corr[Rt^ P), ( Xt-r”M-)^ ] equal to zero for non-negative r. 

4.2 Higher Order Dependency fcr First Order Random Coefficient 
Autoregressive Hodels 

The model to be considered is the first order (p=l) case of (4.1), now 
to be denoted as 

Xt=AtXt_i+Bt, t=0,±l,*2, . . . , (4.6) 

in which Aj- and are independent within and between each t; we also write 

a=E( At ) . M=E( Xt )=( 1-a )-i£( Bt ) . 

Many baisic mathematical and probabilistic properties of this equation have 
been studied by Vervaat (1979). Interest here is restricted mainly to the 
residual crosscovariances of (Rt^/^t-r) t(Xt~M')^#Rt-r] where is the 

first order residual given by 

Rt-Xt-/x-a( (4.7) 

It has been established in Section 4.1 that 

for r=1^2^...^ and so now Cov( ^ Rt-r ) is obtained 

explicitly. 

The calculation begins by writing the required covariance as 

Cov( Rfc2 , Rt_r )=E { [ ( Xfjt )-a( Xt-i-M ) ) ^ [ ( Xt-r"^i )~a( Xt-r-l-^i ) ] } 

=C 2 i( r )-aC 2 i( r +1 )- 2 a{Ci]^i( r )-aCin( r+1 )} 

+a2{C2i( r-1 )-aC2i( r ) } 

where, as defined just before (4.5), 



(4.8) 
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C2i( r )=E{( Xfli )2( Xt-r-M ) ) » CmC r )-E{ ( Xf/i )( Xt-i-/x )( Xt_r-M ) } . 

The higher order triple moment C]n(r) is easily seen to be equivalently 
given by aC 2 i(r-l), in the present instance, and hence (4.8) becomes, 

Cov(Rt2,Rt_r)=C2i(r)-aC2i(r+l)-a2{C2i(r-l)-aC2i(r)), r=l,2, . . . (4.9) 

For r=0, with |X 3 *E(Xt-M-)^ ] » there is the seperate result 

E( Rt^ )*( l+2a3 )M3-3aC2i( 1 ) .- (4.10) 

The calculation of C 2 i(r) is effected by writing the defining equation 
(4.6) in the form 

Xt-ti+H=At( )+( MAt+Bt ) . 

Squaring both sides, multiplying by and taking expectations then 

gives 

C 2 i( r )+2 mC( r )=a 2 C 2 i( r-1 )+2( jia 2 +E( AfcBt ) )C( r-1 ) (4.11) 

where 

C( r )aCov( Xt , X^-r )=a^<j2 , a^=vair( X^ ) , a 2 =E( ) . 

Further simplifications of (4.11) using C(r)=a^a2 gives the recursive 
equation 

C 2 i(r)=a 2 C 2 i(r-l)+b 3 a»^-l, r=l,2 (4.12) 

where 

b3=2a^ {^var( )+Cov( A^ / Bt ) } . 

Equation (4.12) has explicit solution 

C2i(r)=ai2^tj.3+i>3lai2^-a.^)/(3L2-a), r=0,l,... . (4.13) 

Going back to (4.9) for Cov( R^^, Rt-r ) using (4.12), we have finally 
Cov( Rfc2 , Rt_r )=( a2+-a3a2-aa22-a2 )C 2 i( r-1 ) 

+( l-a2+a3-aa2 )b 3 a*^“^ , r=l,2,... . (4.14) 

Thus, (4*14) and (4*13) constitute the e3Q>licit solution for the 
crosscovariances of the residuals (Rt^'^-r)^ notice that when At is the 
constant a, the case of the standard first order linear model, these 
crosscovariances are correctly zero. The NEAR<1) model of Lawrance and 
Lewis (1981) specializes (4.6), by having independence within each (At,Bt) 
and particular forms for At and Bt- 

If the covariances (4.14) are to be converted to correlations, then 
Var(Rt) aJhd Var(Rt^) be obtained, •mis requires the Colbwlng 
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calculation, 

Var(Rt^) = E(Rt*) - [E(Rt^)]^ (4.15) 

where 

E(Rt^) = (1 - a2)a2 = Var(Rt), (4.16) 

E(Rt^*) = (l+a*)|a4 - 4aC3i(l) + 6a2C22(D ~ 4a^Ci3(l), (4.17) 

with 

M4“E[(Xt-M)'^], Ci3(l)=E[Xt-fi)i(Xt_i-M)^]. ( i, j )=( 3, 1 ) , ( 3 , 2 ) , ( I, 3 ) . ( 4. 18 ) 
The joint moments in (4.17) were obtained in terras of the first four 

moments about the mean of the first three moments about zero of and 

the first two moments about zero of These are not reproduced here. 

The explicit result for (4.15) was checked numerically against a slightly 
different method of computations while being used to obtain the 

i 1 lustrations given in Figure 1 and 2 for NEAR( 1 ) models. 

It was remarked in Section 3, following (3.2) that the 

crosscorrelations of (Xt--jx)^ and Rt+r^^^ also be useful in assessing 

higher order dependence. In the present case of first order autoregression 
there are the results, 

Cov[(Xt-M)^, Rt-r] = C 2 i(r) - a C2i(r+1), (4.19) 

Var(Rt) = (l-a^)a^, Var[( Xt-^)^ ) ] = E(Xt-^)^ - a^. (4.20) 

These formula can all be applied to the NEAR(l) model quite simply, 
noting that in this case 

E(At’^) = E(Bt>^) = rl [p4-( l-p)b>^] 

(4.21) 

b = (l-a)/3, p = (l-/3)/[l-(l-a)0]. 



Figures 1 and 2 give the results of computations of Corr[R^^,R^_^] and 
Corr[( X^-M )^^Rt-r ] NEAR( 1 ) model. The top left frame of Figure 1 is 
the lineax FAR(l) case for which all cross-correlations apart from lag zero 
are zero; the other three cases in Figure 1 each have zero 
crosscorrelations at negative lags, in agreement with the theoretical 
results in Section 4.1, but have some non-zero values at the zero and 
positive lags. It is evident that the lag zero cross correlations contain 
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NEAR(l) RESIDUAL CROSSCORRELATIONS (R(T)*2.R(T-R)) 
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Ftgure 1. Four computatton of the crosscorretattons for vartous tags, 
hetueen the ttnear autoregresstve restduat, 

NB^R( 1 ) process utth p(l)-=cnp held constant at 0.75; tn effect the rematntng 
free parameter ts hetng varted through tts attouabte range* 



much of the discriminating information between the four cases; this will be 
so for first order autoregressive models in which much of the higher order 
dependency is at lag one, and the lag zero cross correlation involves both 
third and fourth order aspects of the lag one dependency. For further 
information on the TEAR<1) and PREAR<1) cases see Lawrance and Lewis 

(1981). 

Figure 2 gives Corr[(Xt-M)^.Rt-rl 

Figure 1. negative lags are again zero for all cases, and hence this 
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property does not discriminate the linear from the nonlinear cases. Rather 
it is the strength of the crosscorrelations at positive lags which performs 
this task, albeit less clearly thain the crosscorrelations of the residuals 
auid squared residuals. 



NEAR(l) RESIDUAL CROSSCORREUMIONS ((X(T)-MU)*2.R(T-R)) 



ALPHA-0.999, BEJA-O.TSa. EAR(1) CASE 
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ALPHA=0.758, BaA=0.990, TEAR(1) CASE 




Ftgure 2. Four computattons of the crosscorreVattons , for various tags 
between the linear autoregressive residual, cind for the 
NEAR(l) process with p( 1 held constant at 0.75; in effect the remaining 
free parameter is being varied through its allowable range. 
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4.3 Higher Order Dependency for the Product Autoregreesion liodel 

Another form of nonlinear autoregressive model, called PAR(l), was 
introduced by McKenzie (1982); the basic idea of Its construction is to 
consider the e;q>onentiation of the standard AR(1) equation. Then its 
additive structure becomes multiplicative, and the general form of the 
PAR(l) product autoregressive model model is 

Xt=(Xt-i)^Bt, t=0,±l,±2 (4.22) 

where p is the dependency parauneter, 0^p<l, and (B^) is an independent and 
identically distributed innovation sequence. McKenzie studies the model 
when Bfc is chosen so that (Xt) has a gamma marginal distribution, and 
obtains several basic results; for instance, that p still represents the 
lag one autocorrelation and that the autocorrelations in general satisfy 
the Yule-Walker first order equations ((2.4) with p=l). It is apparent, 
however, that the linear conditional e;q>ectation definition of 
autoregression (equation (2.4)) is not satisfied because of the power form 
implied by (4.22). 

As with the first order rauidora coefficient models in Section 4.2, we 
consider the first order residual R^, given by (4.7), and will likewise 
determine Cov( , Rt-r ) r=0, ±1, ±2, . . . , noting that for this model these 

correlations are non-zero for all lags. We will use the general expression 
(4.8) in terms of the third order central moments C 2 i(r) and Ci^n(r), but 
this time there is no simple relation between them, amd both are needed for 
aai lags. Also, calculation of C 2 i(r) and C 2 n(r) must be in terms of 
their uncentered components, since these are the quantities which can 
immediately be determined from the PAR(1) equation (4.22). The required 
uncentered moments will be written cls 

e2*E(Xt2), e3=E(Xt3) 

e2i(r)=E(Xt2xt_r). eni( r )=E( XtXt-iXfc-r ) (4.23) 

for r-0, 11, i2, . . . , and there is need to note the special causes. 



18 



©21(0 )=^3^ ©lll(0)^2l(l ®lll( 1 )=©2l(“l )• 

All these quantities will be calculated. 

Firsts it is necessary to note the following relations between the 
centred and uncentred moments, 

Clll(0)=C2i( 1 ), Ciii( 1 )=C2i(-l ) (4.24) 

0iii( r )^iii( r C( r )+C{r-l )+C( 1 ) } , ( r?^0, 1 ) . 

The calculations of e 2 i(r) and e^nxir) need to be treated separately 
for positive and negative lags, but follow in the same general manner and 
will be illustrated by that for e 2 i(r) for positive lags. By iterating the 
PAR(1) equation (4.22) r steps backwaxd, 

r-1 

Xt = n 

i=o 

where p( r >spr, r=0,l, . . . . Squaring this equation, multiplying it by X^-r 
and toLking e3q>ec tat ions, gives 

r-1 

e2l(r)=E{Xt_r2^^(»^)+^-) n E{Bt_i2P( i )) . (4.25) 

i=0 

To obtain the e}q>ectations in the repeated product, taking the 2p(i)th 
power of (4.22), leads to 

E{Xt^P^ ^))=E{Xt-i2PP( i))E{Bt2P( 
and hence dropping the unnect^ssary suffix t, to the result 

E{b 2P( i >}=E{ x 2P( i )}/E{x2P( ^+1 ) } . 

Now (4.25) can be expressed purely in moments of X, eis 

e 2 i( r )=E{X2P( )+^ }E( X^ )/E{x2P( ^ ) ) . (4.26) 

To proceed further, invoke a gamma marginal distribution for X with density 
f(x)=e^^x<^le~®*/r(/3), e,/3>0, x>0; (4.27) 

this has mean /3/e, variamce /S/e^, third central moment 20/e^, and there is 
the kth moment result E( X^ )=r( /a+k )/e^r( /3 ) . From (4.26) and similar 
expressions we then have 
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^2l(r) = |0(/3+l)(^2pr)/e3. 



L/3(/3+pl’^l X/s+pli^l+ive^, 



« _ r/3(/3+p)(/>+pr-l+pr)/e3, 

L/3(A+pl^l )(/3+p+p'^l+^)/e3. 



ir~" "L f 2 f • • • 

r=-l,-2, . . . 

r=2 ,3, .. . 

IT X / 2 ^ • 



(4.20) 

(4.29) 



with e 2 i( 0 ), eii_i(0) ei^n(l) being given by the special cases of 

(4.23). Use of (4.24) gives finally the required expressions 

-.r/Q3 



r = /2i3pVe 

Clli(r) [0{(i+0)plr|+l+p2|r 



r=0, 1, . . . 

+p2lrl)/e3, r=-l,-2,... 



l+l_0p|r|-l)/e3 



r=2 , 3 , . . . 



(4.30) 



(4.31) 



r=l,2, , . 

All these may be used in (4.8) to obtain the desired Cov(Rt2,Rt_r ) for 
r=0, ±1, ±2, . . . ; also the simpler covariance or correlation of (Xt~M)2 and 
Rt-r is ecisily available from (4.19) and (4.31). 

Conversion of these covaxiances to correlations requires Var(Rt^) as at 
(4.15); the required intermediate results are 

M4 = 9 /3V©“, 



C3i(l) = 3/3(A+2)p, Ci3(l) = /3{3/3+(l+p)(2+p)}p, 

C 22 U) = /32+2/3pt2/3(/3+2)p2. (4.32) 

These then give the explicit expression 

Var(Rt^) = {( 9/3-1 )/3^ - 4/3(/3+6)p2 + 12Pp^ 

+ /3(9/32-/9+16)p*-12/3p5-4/3p®}/e* (4.33) 

Together with Var(R(-) which is simply (l-p^)0/&^, Cov(Rt^,Rt_r ) can be 
converted into Corr(Rt^/Rt-r )• 

Figures 3 and 4 illustrate the residual and squared residual cross 
correlation for the PAR(l) model. Figure 3 shows how these residual cross 
correlations vary over the range of p values from p=0 to p=0.9 for an 
exponential marginal distribution. At p=0, the PAR(l) is an TDD process 
and the residuals are trivially independent, resulting in the only non-zero 
crosscorrelation at lag zero. As the p value increases the dependence 
spreads out, most strongly at lag minus one and positive lags. An 
interesting feature is the lag zero crosscorrelation which changes from 
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being strongly positive at p«0 to approximately zero at p=0.75, to 
moderately negative at p=0. 

PAR(1) residual crosscorrelations (R(T)*2.R(T-R)) 







LAC(R) 






LAC(R) 




1>C(R) 

Ftgure 3. Computattons of the residual crosscorrelations, Corr(Rf^^,R^^j^) 
for the PAR(l) model. Since 0-1 the marginal distribution is exponential. 
The lag one serial correlation is increased from p( 1)^.0 (upper left) to 
p("i>=0.9 (lower right). 



Figure 4 gives four different gamma cases of the PAR( 1 ) residual cross 
correlations^ all with p=0.75. The gamma shape parameter 0 takes the 
values 0.5^ 1,0, 2,5 and 10.0; these cases indicate that changes in the 
ga mma parameter cause only modest changes in detail of the cross 



correlations. 
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PAR(1) RESIDUAL CROSSCORRELATIONS (R(T)*2.R(T-R)) 

BrTA«0.500. RHO-0.750, THETA* 1.0 BrTA- 1.000, RHO-0.750. THHA- 1 .0 





BrrA=2.500. RHO=0.750. TH£7A=1 0 8ETA= 10.000. RHO = 0.750. THaA=1.0 





Figure 4* Computations of the residual crosscorrelatlons , Corr(R^^,R^^y^) 
for the PAR(l) model. The p for each case of 0.75, and the figures 
Illustrate the effect on the residual crosscorrelatlons of changing the 
Index (3 of the gamma distribution through 0.5, 1.0, 2.5 and 10.0. 



5 . COMCLOSIONS 

A methodology for ainalyzlng higher order dependence in nonlineatr tinie 
series with pth order autoregressive correlation structure hcU3 been 
proposed. It utilizes standaxd uncorrelated linear autoregressive 
residuals, and the crosscorrelation function of these residuals and their 
squares. The behaviour of this crosscorrelation function has been utilized 
for two rather different types of nonlinear model: random coefficient 
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autoregression and multiplicative autoregression: the behaviour has been 
shown to allow discrimination between models in the same clatss with the 
same marginal and autocorrelation structures. 

The residuals crosscorrelation function provides a partial analysis of 
third order Information in the time series; it does not attempt to capture 
all third order information^ which is the aim of such techniques ais 
bispectral analysis and which will often be Intractable with nonlinear 
models. Being based on standard linear residuals, the ainalysis extends 
rather thain replaces conventional residual analysis. 

Developnents of the analysis which focus on the directionality implicit 
in many time series axe being investigated; reversed residuals assume a 
reversed directionality and allow exploration of the consequences of such 
an asssumption. 



Acknowledgements The reseaxch of P.A.W. Lewis was supported by the Office 
of Naval Research under Grant NR-042. The graphs were produced by an 
experimental APL package from IBM which the Naval Postgrciduate School is 
using under an agreement with the IBM Research Centre, Yorktown Heights, 



N.Y. 



23 



6. REFESEMCES 

Andell^ J. (1976) Autoregressive series with random parameters. Math. 
Operattonsforsch . u. Statist., 1, 735-741. 

Granger^ C.W.J. and Andersen^ A.P. (1978) An Introduction to Bilinear Time 
Series Models. Vandenhoeck and Ruprecht, Gottingen. 

Jacobs^ P.A. and bewlB P.A.W. (1983) Stationary discrete autoregressive- 
moving average time series generated by mixtures, J. Time Series 
Analysis, 4, 18-36. 

Jones^ D. (1978) Non-linear autoregressive processes. Proc. Roy. Soc. 
London A, 360, 71-95. 

Lawrance^ A«J. and LewiB^ P«A«W« (1981) A new autoregressive time series 
model in exponential variables. Adv. Appl. Prdb., 13, 86-845. 

Lawrance^ A.J. and Ijewis, P.A^W. (1984) Modelling and residual analysis 
of nonlinear autoregressive time series in exponential variables. 
Preprint . 

liewiB^ P.A.W. (1980) Simple models for positive-valued and discrete-valued 
time series with ARMA correlation structure. In Multivariate Analysls-V 
(Ed. P.R. Krishnaiah), 151-166. North Holland, Amsterdamn. 

Lewis^ P*A«W. (1981) Sixople multivariate time series for simulations of 
complex systmes. In Proc. Winter Simulation Conference (Eds. T. I. Oren, 
C. M. Delfosse and C. M. Shub), 389-390. IEEE Press, New York. 

McKenzie^ E* (1982) Product autoregression: a time series characterization 
of the gamma disribution. J. Appl . Prob., 19, 463-468. 

Hicholls^ D.F. and Q uinn ^ B.G. ( 1982 ) Random Coefficient Autoregressive 
Models: An Introduction. Springer-Verlag, Heidelberg. 

Rosenblatt:^ (1979) Linearity and nonlinearity in time series: 

prediction. Proc. 42nd Session I.S.I. (Manila) , Vol . 1, 423-434. 



- 24 - 



Rosenblatt, M.R. (1980) Linear processes and bispectra. J. Appl. Prob. 17, 
265-270. 

Tjostheim, D., and Paulsen, J. (1989) Bias of some commonly-used time 
series estimates. Biometrika, 70, 389-400. 

Vervaat, W. (1979) On a stochcistic difference equation and a representation 
of non-negative infinitely divisible remdom variables. Adv. Appl. Prob. 
11, 750-783. 

neiss, G. (1975) Time reversibility of linear stocheuatic processes. J. 
Appl. Prob., 12, 831-836. 



DISTRIBUTION LIST 



NO. OF COPIES 

Defense Technical Information Center 2 

Cameron Station 
Alexandria, VA 22314 

Library 2 

Code 0142 

Naval Postgraduate School 
Monterey, CA 93943 

Research Administration 1 

Code 012A 

Naval Postgraduate School 
Monterey, CA 93943 

Library 1 

Code 55 

Naval Postgraduate School 
Monterey, CA 93943 

Professor P.A.V/. Lewis 150 

Code 55Lw 

Naval Postgraduate School 
Monterey, California 93943 



1)213370 



